feat(spend): add net auto-router savings to the cost-optimization dashboard by tin-berri · Pull Request #35521 · BerriAI/litellm

tin-berri · 2026-08-01T23:06:37Z

TLDR

Problem this solves:

The dashboard credits compression and caching, not auto-router model switches
A cold cache means two opposite things and they need opposite math
Charging both alike understates a first turn to ~4% of its value

How it solves it:

Prices each request against the priciest model its hardest tier could pick
Reads the conversation to tell a first turn from a mid-conversation switch
Accrues the dollars into a daily rollup column; never reads LiteLLM_SpendLogs

Relevant issues

Backend half of the auto-router savings driver; the dashboard that renders it is a separate PR stacked on this branch
Supersedes an earlier single PR that carried backend and UI together, and an earlier follow-up that recorded session state in a cache
Rollups, not spend logs: the same read path the compression and caching drivers already use

Linear ticket

Resolves LIT-5046
Resolves LIT-5087

Pre-Submission checklist

I have added meaningful tests
My PR passes all CI/CD checks (e.g., lint, format, unit tests)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Not yet captured against a live proxy, and the gap is stated rather than glossed. What is owed is real auto-routed traffic over two turns of one conversation, then /user/daily/activity showing the first turn's larger autorouter_savings_spend. The available Anthropic key is out of credit.

The arithmetic the change turns on, from the shipped cost map on this branch, 20k prompt and 1k completion, anthropic/claude-opus-5 as baseline against claude-haiku-4-5 selected:

first turn, neither model had anything cached
  write-rate gap   20000 x (6.25e-06 - 1.25e-06)   +0.1000
  output-rate gap   1000 x (2.50e-05 - 5.00e-06)   +0.0200
  total                                            +0.1200

mid-conversation switch, same request
  baseline: opus reads 20k (already warm) + 1k out  0.0350
  actual:   haiku writes 20k (cold) + 1k out        0.0300
  saving                                           +0.0050

a switch by completion length, 20k cached prompt
  completion   200   -0.0110
  completion   500   -0.0050
  completion   750   -0.0000
  completion  1000   +0.0050

Without the discriminator every one of those requests priced at the +0.0050 row, so a first turn that genuinely saved $0.12 could render as a loss.

Type

🆕 New Feature

Changes

Auto-router savings join compression and prompt caching as a third optimization driver, computed per request at spend-write time and accrued into the daily rollup tables. Nothing here reads LiteLLM_SpendLogs; the dashboard reads the same rollups the other two drivers already use.

What it is measured against. Without a router a deployment runs one model, and it has to be one that can carry the hardest request, so the baseline is the priciest model in the router's hardest configured tier. Deriving it from the tier ladder rather than hard-coding a flagship matters: a router choosing only between sonnet and haiku saved nobody the price of opus. A cheap tier is a choice the router made, not a ceiling it was bounded by, which is why the hardest tier and not the whole reachable pool is the candidate set. auto_router_savings_baseline_model overrides it. Candidates are provider-qualified before pricing, since a bare name can resolve to a different vendor's rates or to nothing, and each deployment is priced by its base_model where it has one, which is how Azure deployments are priced everywhere else in the codebase.

How the dollars are computed. Both arms price the request's real usage through generic_cost_per_token, litellm's own cost engine, so tiered rates, ephemeral cache-write tiers and regional uplifts stay consistent with what was actually billed. prompt_tokens already includes the cache buckets, so charging them again at the input rate would price the same tokens twice.

Cache state is the whole problem. The baseline serves every turn, so whether it had the prompt cached is simply whether the conversation was already underway. On a continuing conversation it wrote the prompt on an earlier turn and would only read it now, so this request's write is what switching cost and counts against the saving. On a first turn nothing was cached anywhere for any model; the baseline would have written the same prompt, and both arms carry that write at their own rates. Applying the switch arithmetic to both, which is all a single rollup row can support on its own, understates a first turn badly, and because the write premium is fixed by prompt size while the saving grows with completion length, the understated number can cross zero and report a loss on a request that saved.

Three cache states, not two. A first turn has nothing cached for any model, so both arms write and the saving is the rate gap between them. A switch leaves the chosen model cold while the baseline stays warm on the model it never left, so only the selected arm writes and that write counts against the saving. A turn that stayed on the same model reads most of the prompt and writes only what the conversation grew by; those tokens are new to every model, so the baseline writes them too and its split matches the selected model's exactly. The third case is separated by the request reading more than it wrote rather than reading anything at all, because a switch onto a model already holding a small prefix still writes most of the prompt and must keep paying for it.

Reading the shape instead of remembering it. A second human ask in the message list means an earlier turn was already served. That needs no cache, no session id and no cooperation from the caller, so it covers traffic that sends no session header at all, and it has no failure mode where a cache error masquerades as a first turn. It cannot see a switch on a turn the router never classified, and it reads a few-shot prompt's synthetic turns as prior conversation; both err toward charging the write, which under-claims rather than inflating.

Where the facts travel. The baseline and the conversation shape ride on the existing routing_decision record rather than beside it. That record already goes from the router to the spend log, is already classified for redaction, and is already written-or-cleared per routing attempt, so a fallback that re-enters the hook cannot leave one fact behind to be attributed to a deployment that never routed, and no new key crosses the caller trust boundary.

Why it is signed. Whether a switch pays off is a race between the rate gap and the cache-write cost, and a narrow gap loses. Flooring at zero would leave a number that can only move one way and would hide exactly the routing behaviour an operator needs to see.

Where it lands. A new autorouter_savings_spend column on the six daily rollup tables, declared NotRequired because rows queued by a pod on the previous release carry no such key. It is summed by the rollup merge that the cross-pod Redis drain also runs, and carried through the aggregation query, the per-row accumulation and the response model, so the dashboard reads a value the API actually sends. Regression tests enumerate the drivers from the response model itself and assert each is summed, accumulated, carried and totalled, so a driver added later cannot be half-wired.

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Note

Medium Risk
Touches spend aggregation, DB schema, and pricing logic where incorrect math would misreport savings; changes are heavily tested and fail open to zero on bad inputs.

Overview
Adds a third cost-optimization driver that records signed net dollars from auto-router model choices versus a configured baseline (autorouter_savings_baseline_model), using real request usage and deployment-aware pricing.

Computation extends compute_savings_spend with compute_autorouter_savings, which compares baseline vs selected cost via generic_cost_per_token (optional pre-resolved model_info for per-deployment rates). Cache-aware baseline usage distinguishes first turn, mid-conversation switch, and steady continuation; complexity router attaches conversation_continuing on routing_decision from message history (assistant turn present).

Persistence & API: autorouter_savings_spend on all six daily rollup tables (migration + Prisma), queued as NotRequired for upgrade-safe aggregation, refactored optional-metric upserts in the spend writer, and surfaced through daily activity SQL, SpendMetrics, and dashboard OpenAPI types.

Tests cover savings edge cases (losses, implicit cache pricing, malformed usage) and guard the full read/aggregate path for every *_savings_spend driver.

^{Reviewed by Cursor Bugbot for commit 22795c2. Bugbot is set up for automated code reviews on this repo. Configure here.}

greptile-apps · 2026-08-01T23:09:01Z

Greptile Summary

The PR adds signed auto-router savings to request-time cost calculation and carries the metric through daily spend rollups and activity APIs.

Resolves configured or hardest-tier baseline deployments and records their identity in routing decisions.
Compares baseline and selected deployment costs using cache-aware usage and effective deployment pricing.
Adds autorouter_savings_spend to daily database tables, aggregation queues, API models, and generated UI types.

Confidence Score: 4/5

The PR should not merge until baseline selection handles request-shape-dependent pricing instead of relying on one fixed cache-heavy workload.

A baseline chosen from the fixed 19k-cache-read reference can cease to be the most expensive hardest-tier deployment for uncached prompts or different output lengths, while the selected identity is permanently reused to calculate and persist savings for the actual request.

Files Needing Attention: litellm/router_strategy/savings_baseline.py, litellm/router_strategy/complexity_router/complexity_router.py, litellm/proxy/spend_tracking/savings.py

Important Files Changed

Filename	Overview
litellm/router_strategy/savings_baseline.py	Preserves deployment pricing and includes cache rates when selecting a baseline, but the fixed reference workload can still select the wrong baseline for differently shaped requests.
litellm/proxy/spend_tracking/savings.py	Computes signed, cache-aware savings using effective baseline and selected deployment pricing; it relies on the baseline selected before actual usage is available.
litellm/router_strategy/complexity_router/complexity_router.py	Derives conversation state and records baseline deployment metadata on all routing-decision paths.
litellm/proxy/db/db_spend_update_writer.py	Adds auto-router savings to daily transactions and handles optional additive metrics during mixed-version queue processing.
litellm/proxy/management_endpoints/common_daily_activity.py	Aggregates and exposes auto-router savings alongside the existing optimization metrics.
litellm-proxy-extras/litellm_proxy_extras/migrations/20260731000000_add_autorouter_savings_spend/migration.sql	Adds the non-null, zero-default savings column to all six daily rollup tables.

_{Reviews (6): Last reviewed commit: "fix(spend): rank baseline candidates by ..." | Re-trigger Greptile}

codecov · 2026-08-01T23:12:48Z

Codecov Report

❌ Patch coverage is 88.42975% with 14 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/spend_tracking/savings.py	86.95%	12 Missing ⚠️
litellm/proxy/db/db_spend_update_writer.py	81.81%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

…hboard The dashboard credited compression and prompt caching but said nothing about the optimization that picks the model, so the driver with the largest lever on a bill was the one an operator could not see. Savings are the counterfactual: without a router a deployment runs one model, and it has to be one that can carry the hardest request, so the baseline is the priciest model in the router's hardest configured tier. A cheap tier is a choice the router made, not a ceiling it was bounded by. `auto_router_savings_baseline_model` overrides it for operators who would genuinely have run something else. Both are provider-qualified before pricing, because a bare name can resolve to a different vendor's rates or to nothing at all, and a deployment is priced by its `base_model` where it has one, which is how Azure deployments are priced everywhere else. Both arms price the request's real usage through `generic_cost_per_token` rather than re-deriving per-token arithmetic, so tiered rates, ephemeral cache-write tiers and regional uplifts stay consistent with what was actually billed. `prompt_tokens` already includes the cache buckets, so charging them again at the input rate would price the same tokens twice. Cache state is what makes this hard. The baseline serves every turn, so whether it had the prompt cached is whether the conversation was already underway. On a continuing conversation it wrote the prompt earlier and would only read it now, so this request's write is what switching cost and counts against the saving. On a first turn nothing was cached for any model, the baseline would have written the same prompt, and both arms carry the write at their own rates. Charging the write to both cases understates a first turn to a few percent of its value, and because the write premium is fixed by prompt size while the saving grows with completion length, it can render a profitable route as a loss. That shape is read off the conversation rather than remembered: a second human ask means an earlier turn was served. No cache, no session id, and no dependence on a caller sending a session header. It cannot see a switch on a turn the router did not classify, and it reads a few-shot prompt's synthetic turns as prior conversation; both err toward charging the write, which under-claims. The baseline and the shape ride on the existing `routing_decision` record, which is already carried from the router to the spend log, already classified for redaction, and already written-or-cleared per attempt. A fallback that re-enters the hook therefore cannot leave either fact behind to be attributed to a deployment that never routed, and no new metadata key crosses the trust boundary. The result is signed. Whether a switch pays off is a race between the rate gap and the cache-write cost, and a narrow gap loses; flooring at zero would hide exactly the routing behaviour an operator needs to see. The donut plots only drivers that saved, while the card and range total keep the sign. Savings accrue into a new `autorouter_savings_spend` column on the six daily rollup tables, declared `NotRequired` because rows queued by a pod on the previous release carry no such key. It is summed by the rollup merge the cross-pod Redis drain also runs, and carried through the aggregation query, the per-row accumulation and the response model, so the dashboard reads a value the API actually sends. Tests enumerate the drivers from the response model itself and assert each is summed, accumulated, carried and totalled, so one added later cannot be half-wired.

`_baseline_usage` moved every cache-creation token into the baseline's read bucket whenever the conversation was underway. That is right for a switch, where the baseline never left the model it was on and really would only read, but wrong for a turn that stayed put: the prompt grew, and the tokens written are that growth. They are new to every model, so the baseline would have paid to write them too. Forgiving it that write made the counterfactual cheaper than it was and shrank the reported saving on ordinary steady-state traffic, by about 2% per turn. The selected arm was never involved; it has always been priced on the real usage. The error sat entirely on the baseline. The condition is that the request read more than it wrote, not that it read anything. A switch onto a model already holding a small prefix of this prompt still writes most of it, and that write is the switch's own cost; keying off a nonzero read would have handed such a request the full rate gap, turning +$0.0056 into +$0.1177. Comparing the two buckets separates a warm continuation, which reads far more than it writes, from a cold arrival, which does the reverse, and it leaves the existing invariant intact: a request reading 0 and one reading 1 both still land in the same place.

tin-berri · 2026-08-01T23:55:43Z

@greptile can you review this PR and explain why you gave the score?

codspeed-hq · 2026-08-02T00:37:07Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_autorouter_savings_backend (c4695a4) with litellm_internal_staging (cb8c734)¹}

No successful run was found on litellm_internal_staging (2d1f650) during the generation of this report, so cb8c734 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

…gent turns Two ways the savings number read the wrong thing, both from identifying a model by its name when the name is not what it costs. The counterfactual was ranked and priced on the public rate for the model a deployment names. A deployment may not be charged that rate: the router registers its configured prices under the deployment's own id and deliberately keeps them off the shared model-name key so deployments sharing a backend model do not pollute each other. So a hardest-tier deployment configured above its public rate lost the ranking to a cheaper candidate, and once chosen was priced at a rate nobody pays. Which key prices a deployment is now `_select_model_name_for_cost_calc`'s decision, the resolver the real request is billed through, rather than a second rule here that would have to re-learn that per-second and tiered overrides count, that a partial override still counts, and that a deployment configured at zero is priced at zero rather than treated as unpriced. The arm being subtracted had the same fault and a sharper edge. It priced the spend log's `model`, which on Azure is the deployment name, absent from the cost map, so the whole driver silently read zero for that traffic. It no longer re-derives anything: `model_map_information.model_map_key` is what litellm actually billed the request under, recorded at request time by that same resolver with `base_model` and custom pricing already applied. Separately, the conversation-shape discriminator counted human asks, and an agent loop can run twenty turns on one of them. Its tool traffic rides `tool_result` blocks on user turns that flatten to empty text, and `tool` roles that are never read, so a long agentic conversation looked like its own first turn and was handed the arithmetic that leaves the cache write on both arms. That is the one direction this must never fail in, because it inflates. An assistant turn is the direct evidence that something answered earlier, and it is blind to how the tool plumbing is spelled on either surface.

tin-berri · 2026-08-02T00:41:56Z

@greptile can you review and explain why your score?

tin-berri · 2026-08-02T00:41:58Z

bugbot run

…eeds The served model was resolved through one input at a time, and each choice broke the half the other fixed. `model_map_key` is the served model already resolved through `base_model`, which is the only way an Azure deployment name reaches the cost map at all; without it the selected arm priced a name absent from the map, returned nothing, and the whole driver silently read zero for that traffic. But it is built without `router_model_id`, so it never carries a deployment's own price overrides, and a custom-priced deployment was compared at its public rate while the baseline used the real override. On a deployment configured well above its public rate that inverted the answer outright: a route that lost $21.88 reported saving $0.10. `_select_model_name_for_cost_calc` takes both, so it gets both. Which key prices a deployment stays its decision rather than a rule restated here.

tin-berri · 2026-08-02T01:00:08Z

@greptile can you review, give me a score and explain?

tin-berri · 2026-08-02T01:00:12Z

bugbot run

…oyment The short-circuit compared resolved model identity, so two deployments of one model collapsed to "no switch" and reported zero. They are not the same cost: a deployment can carry a negotiated rate, and routing from the dear one to the list-price one is a real saving the dashboard reported as $0.00 against a true $21.93. Both arms now carry the key litellm prices them under, so the comparison is between deployments rather than between names.

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit cfc4f13. Configure here.}

…e-resolving Four review rounds landed on one mechanism: which identifier prices a deployment. base_model, then the deployment id, then cache-only overrides. Each round added a clause to a resolution rule that should not exist, and a wrong primitive fails once per input shape, so each shape arrived as its own finding. `Router.get_deployment_model_info` already owns this. It merges a deployment's configured prices over the built-in map, folds in `base_model` defaults for deployments whose name is not a model, and falls back to the model name when nothing is overridden. Every shape hand-rolled here (cache-only, partial, per-second, Azure) was that function re-implemented badly. `generic_cost_per_token` now accepts already-resolved rates instead of demanding a name it looks up itself, which is what forced the name-bending in the first place. Both arms resolve through the owner and pass what they got: the counterfactual by the deployment the router would have used, the served request by the deployment that served it. The invented cost-key resolver is gone, and `Baseline` carries a deployment id rather than a key we chose on litellm's behalf. Net 64 insertions against 79 deletions.

veria-ai · 2026-08-02T02:33:30Z

                total_compression_saved_tokens=metadata_metrics.compression_saved_tokens,
                total_compression_savings_spend=metadata_metrics.compression_savings_spend,
                total_prompt_caching_savings_spend=metadata_metrics.prompt_caching_savings_spend,
+                total_autorouter_savings_spend=metadata_metrics.autorouter_savings_spend,


Low: Deployment pricing disclosure

This returns savings calculated from deployment-specific pricing overrides through /user/daily/activity, where non-admin users can read their own traffic. An authenticated user can submit requests with controlled token usage and combine total_spend with this value to infer the baseline deployment's negotiated input and output rates; restrict this metric to administrative responses or avoid using private deployment rates in user-visible analytics.

Declining this one, with the code rather than an argument.

remove_sensitive_info_from_deployment (proxy/common_utils/openai_endpoint_utils.py:13) is the owner of what must not leave the proxy. It pops api_key, client_secret, vertex_credentials, aws_access_key_id and aws_secret_access_key, and deliberately does not touch pricing.

Pricing is published directly, not just inferable. _get_proxy_model_info (proxy_server.py:12912) merges it into /model/info on purpose, comment and all:

# read litellm model_prices_and_context_window.json to get the following: # input_cost_per_token, output_cost_per_token, max_tokens

so a caller reads the rates rather than solving for them, and the dashboard depends on that to show what models cost. Treating the derived figure as sensitive while the endpoint states the inputs would be a posture change for the proxy, not a fix to this PR.

One residual worth naming rather than hiding: the baseline is the hardest tier's deployment, which a given caller may not be entitled to enumerate through /model/info, so for that deployment this is a slightly narrower channel than the endpoint. It stays low: the router can already route that caller's traffic onto those models. Gating the field would also have to live in the model-access resolver, not in the response builder, and the value is summed across callers at write time in the daily rollup, so a viewer-dependent counterfactual is not something a pre-aggregated column can express.

Thanks for the detail. I can't automatically confirm this is safe to dismiss, so I'm leaving the thread open for a maintainer to make the call.

veria-ai · 2026-08-02T02:33:37Z

PR overview

Adds net auto-router savings to the cost-optimization dashboard by including savings calculations in daily activity and spend analytics.

One security issue remains open: authenticated non-admin users may be able to infer deployment-specific negotiated pricing by combining user-visible savings and spend data for controlled requests. This disclosure requires private pricing overrides and carefully chosen traffic, and no issues have yet been addressed.

Open issues (1)

Low: Deployment pricing disclosure — litellm/proxy/management_endpoints/common_daily_activity.py:996

Fixed/addressed: 0 · PR risk: 4/10

…andidates Ranking moved through `Router.get_deployment_model_info`, since what a deployment costs is the router's answer to give; these four cases were still calling the old free-function signature.

tin-berri · 2026-08-03T06:10:42Z

@greptileai can you review, give me a score and explain why

tin-berri · 2026-08-03T06:10:45Z

bugbot run

…two rates "Most expensive" was decided by comparing output rate then input rate. That is a property of a rate, not of a request: a deployment dearer per output token can be cheaper per cached token, so the comparison ordered cache-heavy traffic backwards and recorded the wrong counterfactual. Candidates are now costed on one reference request through the same engine the savings themselves use, which leaves cache read and write rates, tiered tables and every other billing dimension to that engine rather than to another rule restated here. The reference request is cache-heavy because auto-routed traffic is.

tin-berri · 2026-08-03T22:56:58Z

bugbot run

greptile-apps · 2026-08-03T22:59:28Z

Score: 3/5

What's solid:

The core arithmetic is correct. The three-way discrimination (first turn / mid-conversation switch / same-model growth) is sound, and the cache_read > cache_creation threshold for the continuation case is a defensible heuristic with direct test coverage.
The test suite is the strongest part of this PR — it encodes the exact failure modes described in the PR description (test_switching_models_mid_conversation_charges_the_cold_cache_write, test_a_first_turn_is_the_rate_difference_not_a_switch_penalty, test_the_baseline_is_never_charged_a_cache_write, test_an_agent_loop_on_one_human_ask_is_not_a_first_turn). Explicitly testing that signed values survive into SavingsSpend is good.
Migration is safe: ADD COLUMN IF NOT EXISTS ... DEFAULT 0.0 on all six tables, autorouter_savings_spend: NotRequired[float] on BaseDailySpendTransaction. Rolling upgrade is handled correctly.
Graceful degradation throughout — every exception path in savings.py returns 0.0 and logs at debug/warning level; the spend write cannot fail due to this computation.
_cache_creation_token_details is correctly excluded from the baseline PromptTokensDetailsWrapper to prevent generic_cost_per_token from re-charging the phantom write on the baseline side.

Issues preventing 4/5:

1. CI/CD explicitly not passing (checklist). This is a hard blocker regardless of code quality.

2. No proof of fix (checklist). The PR acknowledges this gap; it's a stated requirement.

3. Dead parameter in async_pre_routing_hook signature (complexity_router.py:1400):

async def async_pre_routing_hook(
    ...
    conversation_continuing: bool = True,   # ← never read
) -> PreRoutingHookResponse | None:
    ...
    conversation_continuing = _conversation_is_continuing(...)  # ← immediately overwritten

The default = True on the parameter is unreachable; the computed value always replaces it. Drop the parameter or, if it's meant to allow callers to override, actually use the passed value.

4. _conversation_is_continuing(None) and _conversation_is_continuing([]) both return True (complexity_router.py:253). The PR describes this as conservative, and test_unreadable_messages_stay_conservative confirms it's intentional. Fine as a design choice, but it means single-turn API calls that send messages=None are treated as mid-conversation and have their savings under-claimed. Worth noting in a comment on the if not messages: return True line since it's the non-obvious direction.

Summary: The implementation is architecturally correct, the tests are thorough, and the upgrade path is safe. The blockers are CI/CD and the missing live-traffic proof, not the logic itself.

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit fe4cb35. Configure here.}

…hook It was added to `async_pre_routing_hook` by mistake and immediately overwritten by the value the hook computes, so it never did anything. It also widened a signature every pre-routing strategy shares with the protocol in `types/router.py`, leaving this one router diverged from `AutoRouter` and the interface for no reason. Also records why an unreadable request counts as continuing: no messages is no evidence a turn was served, so it pays the cache write and under-claims rather than being handed a first turn's larger saving on nothing.

tin-berri · 2026-08-03T23:19:53Z

@greptileai can you reply here with score & why

tin-berri · 2026-08-03T23:19:55Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a08e5dc. Configure here.}

greptile-apps · 2026-08-03T23:22:22Z

Score: 4/5

The implementation is well-engineered and the test suite is unusually thorough. Here's the breakdown:

What's strong (pushing toward 5):

Three-state cache discriminator is mathematically correct. _baseline_usage correctly handles first turn (write costs split by rate gap), mid-conversation switch (baseline stays warm, selected arm pays cold write), and continuation (growth tokens are new to every model). Tests verify all three explicitly.
Conservative defaults everywhere — unknown shape → conversation_continuing=True → charges the write → under-claims. Unresolvable model → 0.0. Pricing failure → 0.0. None of these ever block a spend write.
NotRequired on autorouter_savings_spend in BaseDailySpendTransaction is the right upgrade-safety call; older queued rows omit it and every reader coalesces to zero.
Provider normalization via _resolve_model on both arms before comparing is correct — anthropic/claude-opus-5 vs claude-opus-5 canonicalizes to the same identity and correctly returns 0.0.
generic_cost_per_token receiving optional pre-resolved model_info is a clean additive extension — the existing fallback to get_model_info stays, and Azure/deployment-negotiated rates now price correctly rather than resolving to the public rate.
20+ focused tests each pin one behavioral invariant with explicit arithmetic, making regressions visible rather than masked.

What holds it back from 5:

CI not passing. That box is unchecked and the PR acknowledges it. This needs to be green before merge.
No live traffic proof. The entire correctness argument rests on the first-turn/switch discriminator working against real traffic. The PR acknowledges the key is out of credit — fair, but it's still a gap.
_conversation_is_continuing(messages=None) → True is documented as conservative, but a single-turn call (no messages, or explicitly empty) is the dominant case where it's most likely wrong. It under-claims so the sign is right, but the docstring focuses on the general "no messages" case rather than calling out this specific scenario explicitly.
compression_savings_spend and prompt_caching_savings_spend remain Required in BaseDailySpendTransaction while autorouter_savings_spend is NotRequired. Harmless now since the older two are always populated, but a future driver author copying the pattern may not realize the annotation asymmetry is intentional — worth a comment.

The arithmetic is correct, the failure modes are safe, and the test coverage is among the best I've seen for a spend-tracking change. Get CI green and this is ready.

tin-berri · 2026-08-04T00:19:06Z

On the single-turn case: _resolve_messages runs Responses-API input through the translation handler, so chat/completions and both Responses shapes all resolve to one user turn and correctly read continuing=False. Only a request with no readable messages hits that guard, and the router returns None for those, so no routing decision is recorded and the driver reports 0.0 regardless.

surface                               resolved?   continuing
chat/completions, one user msg             True        False
responses API, string input                True        False
responses API, list input                  True        False
embeddings / no messages at all           False         True   <- no decision, savings 0.0

On the annotation asymmetry: a disabled driver writes 0.0 rather than nothing, so this is not about features being off. NotRequired covers a rolling upgrade, where a pod on the previous release queues rows into the shared Redis buffer without the key and a new pod drains them. The older two are Required because they predate that handling, so the honest note is that NotRequired is what a field added after a release needs, not that the contrast is deliberate.

…not price A model with no cache_creation_input_token_cost, which is every OpenAI, Azure and Gemini entry, resolved that rate to 0.0 and carried the whole written prompt for free, so a first turn routed onto a cheaper model reported a loss. Same hole on cache reads. Those tokens are plain input on such a model, so they move into the text bucket.

tin-berri · 2026-08-04T01:06:09Z

@greptileai can you reply here with a score & why

tin-berri · 2026-08-04T01:07:45Z

bugbot run

greptile-apps · 2026-08-04T01:08:58Z

Score: 4/5

The implementation is well-engineered and I'd approve it proceeding to maintainer review with one caveat noted. Here's the breakdown:

What's good:

Architecture is sound. Piggybacks on the existing routing_decision flow, reuses generic_cost_per_token, writes only to rollup tables. Nothing new in the critical read path.
Backward compatibility is handled correctly. NotRequired[float] on BaseDailySpendTransaction, ADD COLUMN IF NOT EXISTS in the migration, and missing-key-coalesces-to-zero at every aggregation point mean a rolling upgrade won't corrupt rows queued by the previous release.
The three-case cache discriminator is the hard part, and it's right. The warm condition (conversation_continuing and cache_creation > 0 and cache_read <= cache_creation) correctly separates a cold switch from ordinary conversation growth, and the _baseline_cache_rate_keys fallback correctly handles models (OpenAI, Azure, Gemini) that carry no explicit cache-write rate.
conversation_continuing=True as the conservative default — unknown shapes under-claim rather than inflate. Correct direction for a trust boundary.
Signed savings with no zero floor. The dashboard must be able to report a losing switch; flooring here would mask exactly the routing behaviour operators need to see.
Test coverage is exceptional — 20+ targeted tests covering multimodal, implicit caching, cross-provider pricing, first-turn vs switch, and the "mostly cold" edge case. The test_every_optional_daily_metric_aggregates meta-test that reflects over BaseDailySpendTransaction annotations is a particularly good regression net.
optional_metrics dict refactor in _update_daily_spend is a real improvement — new drivers now require a one-line addition instead of a copied if block.

What's missing for a 5:

No live-traffic proof. The PR itself notes the Anthropic key is out of credit and the screenshots section is empty. The two unchecked CI boxes (passing tests, Greptile score ≥ 4/5) are the stated gates. The math checks out analytically and the unit tests pass locally, but the end-to-end path — router → routing_decision → spend write → rollup → /user/daily/activity response — has no captured trace validating that autorouter_savings_spend propagates non-zero to the API response. This is the gap to close before merge.
Minor: _get_llm_router() runs on every spend write regardless of whether autorouter_savings_baseline_model is configured. Since compute_savings_spend exits early when decision is falsy, the router is fetched and immediately discarded on unrouted requests. Not a correctness issue, but a small unconditional overhead in the write path.
Minor: _resolve_messages is called twice in async_pre_routing_hook — once for _conversation_is_continuing and implicitly again inside _classify_and_route. Worth caching the result on a local variable.

Verdict: The core logic, backward-compat story, and test suite are all in good shape. The blocker is the missing end-to-end trace. Once that's captured (or the CI checks go green on a proxy integration test), this is ready.

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 22795c2. Configure here.}

mateo-berri

LGTM. Thanks!

mateo-berri · 2026-08-04T02:33:03Z


+                                    # Additive metrics that older queued rows may omit; one
+                                    # enumeration feeds both the create and the increment below
+                                    optional_metrics = {


nit: I would wrap in a MappingProxyType if you're not mutating it. That way, it's a type error if you try to mutate it unexpectedly

Strongly prefer constructing the dict in one place all at once and marking it as unchangeable, instead of slowly adding to it over time, as the latter can lead to a whole class of "wait, did someone mutate this out from under me?" bugs. AFAICT, you've done pt. 1 here but not pt. 2

Even better if they can be frozen slots=True dataclasses, because of the typing guarantees but that's another discussion

Took the one-shot half, skipped the freeze, because prisma rejects it.

Both payloads go straight to table.upsert, and prisma's builder branches on isinstance(value, dict) to tell a nested node from a scalar (builder.py:851,903). A mappingproxy is a Mapping but not a dict, so it falls to the serializer:

plain dict -> OK MappingProxyType -> TypeError: Type <class 'mappingproxy'> not serializable

That would raise inside the batch upsert, where the surrounding except Exception as batch_error logs it and moves on, so daily rollups would quietly stop updating. Same blocker kills the frozen-dataclass version, plus it would need a to_dict() at the prisma boundary and hand the guarantee straight back.

Your pt. 2 was right though, and worse than the nit suggested: common_data and update_data were both appended to after construction (request_id for tag rows, then endpoint unconditionally). The conditional key is now resolved to a spreadable value first, so both are single literals and the tag branch appears once instead of twice. Verified the payloads are byte-identical across user/tag and both endpoint shapes.

optional_metrics is the one that could be wrapped safely, since it is only ever spread. Happy to if you want the marker somewhere, though it is already built in one shot and never touched.

it would need a to_dict() at the prisma boundary and hand the guarantee straight back

What do you mean "hand the guarantee straight back"? I understand we might need to convert to dict at the prisma boundary, but I would argue that doesn't defeat the point of frozen dataclasses. All the dangerous stuff can be put in simple seam functions (e.g., see the repository classes in our codebase) while core heavy-duty business logic is stateless, pure, and strongly typed

mateo-berri · 2026-08-04T02:33:21Z

+                                    }
+
                                    # Common data structure for both create and update
                                    common_data = {


Same here. I would approach this with all dicts

Same change here: update_data is now a single literal too, with endpoint and the tag request_id folded in rather than appended after. Not frozen, for the prisma reason in the thread above.

`common_data` and `update_data` were constructed and then appended to: `request_id` conditionally for tag rows, `endpoint` unconditionally a few lines later. A dict that grows after its literal cannot be reasoned about by reading the literal, which is the whole point of building it at once. The conditional key resolves to a spreadable value before either payload, so both are single expressions and the tag branch appears once instead of twice. Not wrapped in MappingProxyType, though it was suggested: these go straight to prisma, whose query builder branches on `isinstance(value, dict)` to tell a nested node from a scalar. A mappingproxy is a Mapping but not a dict, so it falls through to the serializer and raises `TypeError: Type <class 'mappingproxy'> not serializable` inside the batch upsert, where the surrounding except would log it and leave the rollups silently unwritten.

…ne budget Building both payloads as single literals traded a mutation for two dict literals, and LIT002 counts construction rather than mutation, so the change the review asked for is the one the gate charges for. The empty branch is the avoidable half: it is the same value every time, so it moves to a module constant built once instead of a literal per transaction, and it is a read-only mapping so none of the call sites that spread it can fill it in later.

…ardest tier (#35907) * feat(spend): derive a default auto-router savings baseline from the hardest tier The savings driver shipped off by default: unless an operator names litellm_settings.autorouter_savings_baseline_model, every auto-routed request records $0.00 and the dashboard card never populates. Nobody discovers a knob whose feature they have never seen work, so the default has to come from somewhere the proxy already knows. The router's own tier ladder is that place. Without a router a deployment runs one model that can carry the hardest request it will see, so the derived baseline is the priciest model in the hardest configured tier, REASONING when present, otherwise the most severe tier the router actually defines. A cheap tier is a choice the router made, not a ceiling it was bounded by. An earlier draft of #35521 derived this per request and was deleted for it: ranking candidates against the request that ran meant reading the request, and every input shape it could take produced its own review finding. This derivation is ranked against one fixed reference request instead, a cache-heavy shape matching real auto-routed traffic, so it never reads the request at all. Candidates still resolve through the router's deployments, so Azure base_model and per-deployment pricing overrides rank correctly. The deciding router records the result on its routing_decision, because one model name can carry several tag-scoped routers with different tier ladders and only the deciding instance knows which of them routed the request. The spend writer's precedence is: configured baseline, then the recorded one, then off. When the setting is present the router skips deriving entirely rather than pricing candidates per decision only to be ignored. Resolution never raises; an unresolvable baseline zeroes the driver instead of failing a live request. Rows queued by a pod on the previous release carry no recorded baseline and fall back to the configured setting, exactly as today. The schema.d.ts regeneration also picks up the reminder_markers field that UI-19232 (#35874) added without regenerating, so one hunk there is inherited staleness rather than part of this change. * fix(spend): cache the derived baseline, price it by deployment, keep it out of the routing preview Three review findings on the derived baseline, addressed together because they all sit on the same value's path from derivation to consumer. Derivation walked and priced the hardest tier's whole pool inside a property read on every routing decision, unbounded by pool size. The router now caches the result per instance with a 30 second TTL, None results included, so the hot path is a clock compare and a deployment edit still lands within a window no operator watches closer than. Ranking used each deployment's effective pricing but recorded only the model name, so the spend writer priced the winning baseline at its public rate: a hardest tier whose deployment carries a negotiated rate produced materially wrong savings. The decision now also records savings_baseline_deployment_id and the writer resolves it through Router.get_deployment_model_info, exactly as the selected arm already does. The id is ignored whenever the configured setting overrides the recorded baseline, since the setting names a model, not a deployment. /auto_router/test_routing returns the routing decision verbatim to team admins while only authorizing the classifier and embedding models, so a derived baseline would resolve another team's model-group alias into its backend provider/model mapping and hand it to a caller never authorized for it. The preview's throwaway router is built with derive_savings_baseline=False; its decisions are never spend-tracked, so nothing is lost, and a source-pinning test keeps the flag on the endpoint. Also strips the explanatory comments this PR had added. * refactor(spend): pin the derived baseline per router instance instead of a TTL Creating or editing a router already rebuilds its ComplexityRouter instance, through unregister and re-add on upsert and through the registry reset on a full model_list load, so a value derived once per instance refreshes on exactly the flows that can change it. That makes the TTL a solution to a problem the rebuild lifecycle already solves, and it goes. Derivation stays deferred to first use rather than running in __init__: during a config load this router can be constructed before the deployments its tiers name, and a baseline pinned at that moment would be empty for the process lifetime. The one behavior the TTL had that the pin does not: editing a tier deployment without touching the router itself refreshed the baseline within a window. That edit path rebuilds only the edited deployment's own strategies, so the pin holds the old answer until the router is next saved or the config next loads. A stale deployment id degrades to public-rate pricing rather than failing, which is where every other unresolvable baseline already lands.

…ardest tier (BerriAI#35907) * feat(spend): derive a default auto-router savings baseline from the hardest tier The savings driver shipped off by default: unless an operator names litellm_settings.autorouter_savings_baseline_model, every auto-routed request records $0.00 and the dashboard card never populates. Nobody discovers a knob whose feature they have never seen work, so the default has to come from somewhere the proxy already knows. The router's own tier ladder is that place. Without a router a deployment runs one model that can carry the hardest request it will see, so the derived baseline is the priciest model in the hardest configured tier, REASONING when present, otherwise the most severe tier the router actually defines. A cheap tier is a choice the router made, not a ceiling it was bounded by. An earlier draft of BerriAI#35521 derived this per request and was deleted for it: ranking candidates against the request that ran meant reading the request, and every input shape it could take produced its own review finding. This derivation is ranked against one fixed reference request instead, a cache-heavy shape matching real auto-routed traffic, so it never reads the request at all. Candidates still resolve through the router's deployments, so Azure base_model and per-deployment pricing overrides rank correctly. The deciding router records the result on its routing_decision, because one model name can carry several tag-scoped routers with different tier ladders and only the deciding instance knows which of them routed the request. The spend writer's precedence is: configured baseline, then the recorded one, then off. When the setting is present the router skips deriving entirely rather than pricing candidates per decision only to be ignored. Resolution never raises; an unresolvable baseline zeroes the driver instead of failing a live request. Rows queued by a pod on the previous release carry no recorded baseline and fall back to the configured setting, exactly as today. The schema.d.ts regeneration also picks up the reminder_markers field that UI-19232 (BerriAI#35874) added without regenerating, so one hunk there is inherited staleness rather than part of this change. * fix(spend): cache the derived baseline, price it by deployment, keep it out of the routing preview Three review findings on the derived baseline, addressed together because they all sit on the same value's path from derivation to consumer. Derivation walked and priced the hardest tier's whole pool inside a property read on every routing decision, unbounded by pool size. The router now caches the result per instance with a 30 second TTL, None results included, so the hot path is a clock compare and a deployment edit still lands within a window no operator watches closer than. Ranking used each deployment's effective pricing but recorded only the model name, so the spend writer priced the winning baseline at its public rate: a hardest tier whose deployment carries a negotiated rate produced materially wrong savings. The decision now also records savings_baseline_deployment_id and the writer resolves it through Router.get_deployment_model_info, exactly as the selected arm already does. The id is ignored whenever the configured setting overrides the recorded baseline, since the setting names a model, not a deployment. /auto_router/test_routing returns the routing decision verbatim to team admins while only authorizing the classifier and embedding models, so a derived baseline would resolve another team's model-group alias into its backend provider/model mapping and hand it to a caller never authorized for it. The preview's throwaway router is built with derive_savings_baseline=False; its decisions are never spend-tracked, so nothing is lost, and a source-pinning test keeps the flag on the endpoint. Also strips the explanatory comments this PR had added. * refactor(spend): pin the derived baseline per router instance instead of a TTL Creating or editing a router already rebuilds its ComplexityRouter instance, through unregister and re-add on upsert and through the registry reset on a full model_list load, so a value derived once per instance refreshes on exactly the flows that can change it. That makes the TTL a solution to a problem the rebuild lifecycle already solves, and it goes. Derivation stays deferred to first use rather than running in __init__: during a config load this router can be constructed before the deployments its tiers name, and a baseline pinned at that moment would be empty for the process lifetime. The one behavior the TTL had that the pin does not: editing a tier deployment without touching the router itself refreshed the baseline within a window. That edit path rebuilds only the edited deployment's own strategies, so the pin holds the old answer until the router is next saved or the config next loads. A stale deployment id degrades to public-rate pricing rather than failing, which is where every other unresolvable baseline already lands.

…7.0) (#336) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.96.2` → `v1.97.0` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.97.0`](https://github.com/BerriAI/litellm/releases/tag/v1.97.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.97.0...v1.97.0) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.97.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.97.0/cosign.pub \ ghcr.io/berriai/litellm:v1.97.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - feat(proxy): resolve Cursor thinking/fast model-name suffixes on /cursor/chat/completions by [@mateo-berri](https://github.com/mateo-berri) in [#35554](https://github.com/BerriAI/litellm/pull/35554) - fix(team-callbacks): actually stop logging when disable\_logging is called by [@yucheng-berri](https://github.com/yucheng-berri) in [#35520](https://github.com/BerriAI/litellm/pull/35520) - refactor(lint): drop redundant !s f-string conversion flags and fix displaced import-group comments by [@mateo-berri](https://github.com/mateo-berri) in [#35546](https://github.com/BerriAI/litellm/pull/35546) - fix(proxy): backfill null user\_email on existing users during JWT auth by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#34588](https://github.com/BerriAI/litellm/pull/34588) - feat(playground): add non-streaming response toggle by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35560](https://github.com/BerriAI/litellm/pull/35560) - feat(teams): apply default organization to new teams from default team settings by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35540](https://github.com/BerriAI/litellm/pull/35540) - fix(ui): block Playground page for viewer roles on direct URL access by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35676](https://github.com/BerriAI/litellm/pull/35676) - fix(caching): close evicted LLM clients so their connections are reclaimed by [@yassin-berriai](https://github.com/yassin-berriai) in [#35492](https://github.com/BerriAI/litellm/pull/35492) - chore(deps): update brace-expansion, postcss, and gitpython to current patch releases by [@yuneng-berri](https://github.com/yuneng-berri) in [#35692](https://github.com/BerriAI/litellm/pull/35692) - refactor(ui): rename the create MCP server component to PascalCase by [@yuneng-berri](https://github.com/yuneng-berri) in [#35686](https://github.com/BerriAI/litellm/pull/35686) - fix(openai): drop undefined Union from owns\_wrapped\_http\_client annotation by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35706](https://github.com/BerriAI/litellm/pull/35706) - fix(openai): drop the undefined Union from owns\_wrapped\_http\_client by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35704](https://github.com/BerriAI/litellm/pull/35704) - chore(ui): note Google's Agent Platform rename in vector store setup by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28076](https://github.com/BerriAI/litellm/pull/28076) - fix(proxy): apply key/team router\_settings.model\_group\_alias by [@yassin-berriai](https://github.com/yassin-berriai) in [#35486](https://github.com/BerriAI/litellm/pull/35486) - feat(complexity\_router): default session affinity off and expose it in the UI by [@tin-berri](https://github.com/tin-berri) in [#35714](https://github.com/BerriAI/litellm/pull/35714) - fix(datadog): read team callback dd\_\* params from kwargs instead of blocked dynamic params ([#35115](https://github.com/BerriAI/litellm/issues/35115) port) by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35687](https://github.com/BerriAI/litellm/pull/35687) - refactor(ui): extract the MCP create form's logic and field groups by [@yuneng-berri](https://github.com/yuneng-berri) in [#35694](https://github.com/BerriAI/litellm/pull/35694) - test(ui): tier the MCP create tests into unit and integration by [@yuneng-berri](https://github.com/yuneng-berri) in [#35697](https://github.com/BerriAI/litellm/pull/35697) - fix(proxy): redact credential headers from request logging copies by [@yucheng-berri](https://github.com/yucheng-berri) in [#35678](https://github.com/BerriAI/litellm/pull/35678) - feat(guardrails/rubrik): prompt moderation, response-text blocking, streaming buffer, failure logging by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35722](https://github.com/BerriAI/litellm/pull/35722) - fix(ui): render Responses API request and response in the logs drawer by [@yuneng-berri](https://github.com/yuneng-berri) in [#35718](https://github.com/BerriAI/litellm/pull/35718) - fix(ui): hide guardrail review buttons from non-admin users by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#27535](https://github.com/BerriAI/litellm/pull/27535) - feat(team): custom metadata validation hook for team create and update by [@yuneng-berri](https://github.com/yuneng-berri) in [#33353](https://github.com/BerriAI/litellm/pull/33353) - ci(circleci): install a pinned Rust toolchain on the Linux jobs by [@yuneng-berri](https://github.com/yuneng-berri) in [#35519](https://github.com/BerriAI/litellm/pull/35519) - fix(bedrock): stop forwarding no-op toolSpec.strict to Converse by [@tin-berri](https://github.com/tin-berri) in [#35688](https://github.com/BerriAI/litellm/pull/35688) - fix(ui): reject an auto-router keyword rule left empty instead of dropping it by [@tin-berri](https://github.com/tin-berri) in [#35705](https://github.com/BerriAI/litellm/pull/35705) - fix(guardrails/rubrik): attribute blocked requests to the caller that made them by [@yucheng-berri](https://github.com/yucheng-berri) in [#35734](https://github.com/BerriAI/litellm/pull/35734) - fix(responses): forward client headers to the provider on /v1/responses by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#34531](https://github.com/BerriAI/litellm/pull/34531) - feat(spend): add net auto-router savings to the cost-optimization dashboard by [@tin-berri](https://github.com/tin-berri) in [#35521](https://github.com/BerriAI/litellm/pull/35521) - chore(typing): clear basedpyright Any errors in budget reset, access groups, and cache settings by [@mateo-berri](https://github.com/mateo-berri) in [#35719](https://github.com/BerriAI/litellm/pull/35719) - fix(spend): read what a request cost from the record instead of pricing it again by [@tin-berri](https://github.com/tin-berri) in [#35736](https://github.com/BerriAI/litellm/pull/35736) - perf: install hiredis so redis-py parses replies with its C parser by [@Classic298](https://github.com/Classic298) in [#35709](https://github.com/BerriAI/litellm/pull/35709) - feat(ui): show auto-router savings on the cost-optimization dashboard by [@tin-berri](https://github.com/tin-berri) in [#35522](https://github.com/BerriAI/litellm/pull/35522) - perf: build log messages lazily so filtered-out log records cost nothing by [@Classic298](https://github.com/Classic298) in [#35703](https://github.com/BerriAI/litellm/pull/35703) - fix(proxy): retry model cost map fetch with Retry-After-aware backoff and keep current map on reload failure by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35739](https://github.com/BerriAI/litellm/pull/35739) - feat(otel): stamp service tier attributes on inference spans by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35679](https://github.com/BerriAI/litellm/pull/35679) - fix(proxy): log the model cost map reload failure lazily by [@tin-berri](https://github.com/tin-berri) in [#35750](https://github.com/BerriAI/litellm/pull/35750) - fix(groq): translate web\_search\_options to the browser\_search tool by [@hMED22](https://github.com/hMED22) in [#34971](https://github.com/BerriAI/litellm/pull/34971) - feat(ui): add admin-configurable user banner by [@yuneng-berri](https://github.com/yuneng-berri) in [#35729](https://github.com/BerriAI/litellm/pull/35729) - fix(e2e): make spend-counter redis connection env-driven for non-cluster deployments by [@yuneng-berri](https://github.com/yuneng-berri) in [#35732](https://github.com/BerriAI/litellm/pull/35732) - fix(proxy): make /cursor/chat/completions work with Cursor agent mode by [@tin-berri](https://github.com/tin-berri) in [#34029](https://github.com/BerriAI/litellm/pull/34029) - fix(proxy): propagate user\_email and bind api\_key on JWT auth attribution paths by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#34331](https://github.com/BerriAI/litellm/pull/34331) - chore(build): move the Admin UI toolchain to Node 24 by [@yuneng-berri](https://github.com/yuneng-berri) in [#35801](https://github.com/BerriAI/litellm/pull/35801) - test(e2e): vendor API strategy coverage across endpoints by [@mubashir1osmani](https://github.com/mubashir1osmani) in [#34649](https://github.com/BerriAI/litellm/pull/34649) - chore(deps): upgrade cryptography to 50.0.0 by [@yuneng-berri](https://github.com/yuneng-berri) in [#35803](https://github.com/BerriAI/litellm/pull/35803) - test(e2e): cover legacy text /completions endpoint by [@mubashir1osmani](https://github.com/mubashir1osmani) in [#34431](https://github.com/BerriAI/litellm/pull/34431) - feat(gemini): add gemini-robotics-er-2-preview and gemini-robotics-er-1.6-preview by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35555](https://github.com/BerriAI/litellm/pull/35555) - test(e2e): move load/perf testing out of the main suite and drop the vllm passthrough test by [@yuneng-berri](https://github.com/yuneng-berri) in [#35820](https://github.com/BerriAI/litellm/pull/35820) - feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) by [@mateo-berri](https://github.com/mateo-berri) in [#35807](https://github.com/BerriAI/litellm/pull/35807) - chore: bump litellm-proxy-extras 0.4.81 -> 0.4.82, litellm 1.96.0 -> 1.97.0 by [@yuneng-berri](https://github.com/yuneng-berri) in [#35810](https://github.com/BerriAI/litellm/pull/35810) - fix(bedrock): drop conflicting tool\_choice.type when toolConfig.toolChoice is set by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35738](https://github.com/BerriAI/litellm/pull/35738) - docs(CLAUDE.md): prefer commas over semicolons when replacing em dashes by [@mateo-berri](https://github.com/mateo-berri) in [#35825](https://github.com/BerriAI/litellm/pull/35825) - chore(lint): zero out basedpyright headroom for purely local rules by [@mateo-berri](https://github.com/mateo-berri) in [#35828](https://github.com/BerriAI/litellm/pull/35828) - test(e2e): retry provider-transient statuses at the transport with bounded backoff by [@yuneng-berri](https://github.com/yuneng-berri) in [#35824](https://github.com/BerriAI/litellm/pull/35824) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#35836](https://github.com/BerriAI/litellm/pull/35836) - refactor(ui): route MCP session tokens through the shared storage helper by [@yuneng-berri](https://github.com/yuneng-berri) in [#35835](https://github.com/BerriAI/litellm/pull/35835) - docs(helm): replace the classic chart's 128Mi resource example with the documented 4Gi sizing by [@yassin-berriai](https://github.com/yassin-berriai) in [#35830](https://github.com/BerriAI/litellm/pull/35830) - fix(proxy): persist periodic reload schedule state so status survives restarts and fires without store\_model\_in\_db by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35165](https://github.com/BerriAI/litellm/pull/35165) - fix(router): eagerly fetch Vertex AI deferred stream to surface HTTP errors in \_acompletion fallback path by [@deepanshululla](https://github.com/deepanshululla) in [#34627](https://github.com/BerriAI/litellm/pull/34627) - fix(azure\_storage): honor AZURE\_STORAGE\_ENDPOINT\_SUFFIX for sovereign clouds by [@yucheng-berri](https://github.com/yucheng-berri) in [#35806](https://github.com/BerriAI/litellm/pull/35806) - fix(proxy): apply key\_alias/key\_hash filters to all /key/list visibility branches by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35840](https://github.com/BerriAI/litellm/pull/35840) - fix(proxy): enforce per-model budgets against resolved cursor model variants by [@mateo-berri](https://github.com/mateo-berri) in [#35834](https://github.com/BerriAI/litellm/pull/35834) - feat(ui): reorder Add Auto Router into name + template, with a collapsible detailed config by [@tin-berri](https://github.com/tin-berri) in [#35746](https://github.com/BerriAI/litellm/pull/35746) - test: repair three failing suites on litellm\_internal\_staging by [@yuneng-berri](https://github.com/yuneng-berri) in [#35845](https://github.com/BerriAI/litellm/pull/35845) - fix(guardrails): scan model output on the /openai/v1/responses alias by [@yucheng-berri](https://github.com/yucheng-berri) in [#35818](https://github.com/BerriAI/litellm/pull/35818) - ci: pin Node on the Playwright UI lanes so npm ci meets the engines floor by [@yuneng-berri](https://github.com/yuneng-berri) in [#35848](https://github.com/BerriAI/litellm/pull/35848) - fix(pricing): apply OpenAI's gpt-5.6 terra/luna cut to Azure cost map by [@mubashir1osmani](https://github.com/mubashir1osmani) in [#35481](https://github.com/BerriAI/litellm/pull/35481) - feat(spend): add caller-scoped key/user/team/organization spend report endpoints by [@yuneng-berri](https://github.com/yuneng-berri) in [#35725](https://github.com/BerriAI/litellm/pull/35725) - revert: "fix(caching): close evicted LLM clients so their connections are reclaimed ([#35492](https://github.com/BerriAI/litellm/issues/35492))" by [@mateo-berri](https://github.com/mateo-berri) in [#35856](https://github.com/BerriAI/litellm/pull/35856) - refactor(repositories): add prisma protocol seams and a spend-reset unit of work by [@mateo-berri](https://github.com/mateo-berri) in [#35748](https://github.com/BerriAI/litellm/pull/35748) - perf(streaming): assemble streamed tool-call arguments in linear time by [@mateo-berri](https://github.com/mateo-berri) in [#35826](https://github.com/BerriAI/litellm/pull/35826) - fix(s3\_v2): sign S3 object URLs with S3SigV4Auth so encoded paths verify by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35726](https://github.com/BerriAI/litellm/pull/35726) - test(e2e): self-seed the ui suite's password-login users in global setup by [@yuneng-berri](https://github.com/yuneng-berri) in [#35863](https://github.com/BerriAI/litellm/pull/35863) - fix(claude-code): create-only skill registration with a PUT update route (LIT-4110) by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#31752](https://github.com/BerriAI/litellm/pull/31752) - fix(proxy): fix zguard httpcode when block input by [@jwang-gif](https://github.com/jwang-gif) in [#31948](https://github.com/BerriAI/litellm/pull/31948) - fix(lint): pick the merge-aware base so in-progress merges are not blamed for base drift by [@mateo-berri](https://github.com/mateo-berri) in [#35868](https://github.com/BerriAI/litellm/pull/35868) - chore: bump litellm-proxy-extras 0.4.82 -> 0.4.83 by [@yuneng-berri](https://github.com/yuneng-berri) in [#35877](https://github.com/BerriAI/litellm/pull/35877) - feat(ui): add Test Routing to the auto router create form by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35859](https://github.com/BerriAI/litellm/pull/35859) - fix(ui): derive auto-router preset tests from the bundled preset JSON by [@tin-berri](https://github.com/tin-berri) in [#35882](https://github.com/BerriAI/litellm/pull/35882) - revert: "test(e2e): vendor API strategy coverage across endpoints" ([#34649](https://github.com/BerriAI/litellm/issues/34649)) by [@yuneng-berri](https://github.com/yuneng-berri) in [#35881](https://github.com/BerriAI/litellm/pull/35881) - chore(deps): bump grpc and golang.org/x modules in the terraform provider by [@yuneng-berri](https://github.com/yuneng-berri) in [#35844](https://github.com/BerriAI/litellm/pull/35844) - test(e2e): skip view-backed global spend probes pending LIT-5211 by [@yuneng-berri](https://github.com/yuneng-berri) in [#35875](https://github.com/BerriAI/litellm/pull/35875) - fix(lint): move the basedpyright heap flag into the type check gate by [@mateo-berri](https://github.com/mateo-berri) in [#35869](https://github.com/BerriAI/litellm/pull/35869) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#35876](https://github.com/BerriAI/litellm/pull/35876) - feat(ui): add role capability gating, migrate Tool Policies route by [@yuneng-berri](https://github.com/yuneng-berri) in [#35812](https://github.com/BerriAI/litellm/pull/35812) - refactor(ui): inject the fetch client's base url instead of reading it at import by [@yuneng-berri](https://github.com/yuneng-berri) in [#35802](https://github.com/BerriAI/litellm/pull/35802) - chore: remove unused .flake8 config and flake8 dev dependency by [@mateo-berri](https://github.com/mateo-berri) in [#35888](https://github.com/BerriAI/litellm/pull/35888) - chore: stop advising pre-commit and bootstrap by [@mateo-berri](https://github.com/mateo-berri) in [#35884](https://github.com/BerriAI/litellm/pull/35884) - fix(auth): name enable\_jwt\_auth when a JWT-shaped key is rejected by [@yassin-berriai](https://github.com/yassin-berriai) in [#35831](https://github.com/BerriAI/litellm/pull/35831) - feat(auto-router): make reminder marker pair configurable by [@akapur99](https://github.com/akapur99) in [#35874](https://github.com/BerriAI/litellm/pull/35874) - fix(UI): update anthropic model presets by [@tin-berri](https://github.com/tin-berri) in [#35896](https://github.com/BerriAI/litellm/pull/35896) - fix(bootstrap): switch to the dashboard node floor via nvm or fnm by [@mateo-berri](https://github.com/mateo-berri) in [#35895](https://github.com/BerriAI/litellm/pull/35895) - perf(pre-commit): run python, dashboard, and gen-api checks concurrently by [@mateo-berri](https://github.com/mateo-berri) in [#35903](https://github.com/BerriAI/litellm/pull/35903) - feat(spend): derive a default auto-router savings baseline from the hardest tier by [@tin-berri](https://github.com/tin-berri) in [#35907](https://github.com/BerriAI/litellm/pull/35907) - fix(http\_handler): self-heal handler clients closed after cache eviction by [@mateo-berri](https://github.com/mateo-berri) in [#35862](https://github.com/BerriAI/litellm/pull/35862) - fix(cost\_tracking): keep OpenAI prompt cache token details through usage reassembly by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#34812](https://github.com/BerriAI/litellm/pull/34812) - fix(cost): bill gpt-5.6 prompt cache reads at the cache read rate by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#34957](https://github.com/BerriAI/litellm/pull/34957) - fix(batches): account for Responses API usage by [@rimysore](https://github.com/rimysore) in [#35367](https://github.com/BerriAI/litellm/pull/35367) - ci: retry Codecov uploads and stop failing jobs on OIDC token flakes by [@mateo-berri](https://github.com/mateo-berri) in [#35251](https://github.com/BerriAI/litellm/pull/35251) - feat(complexity\_router): let operators rename the four complexity tiers by [@akapur99](https://github.com/akapur99) in [#35893](https://github.com/BerriAI/litellm/pull/35893) - chore(lint): zero stale ruff and LIT headroom and strip inert type: ignore comments by [@mateo-berri](https://github.com/mateo-berri) in [#35928](https://github.com/BerriAI/litellm/pull/35928) - chore(lint): zero out seven more purely local basedpyright rules by [@mateo-berri](https://github.com/mateo-berri) in [#35927](https://github.com/BerriAI/litellm/pull/35927) - chore(ui): zero stale headroom on local dashboard eslint budgets by [@mateo-berri](https://github.com/mateo-berri) in [#35929](https://github.com/BerriAI/litellm/pull/35929) - fix(managed-files): skip rows without file objects by [@rimysore](https://github.com/rimysore) in [#35365](https://github.com/BerriAI/litellm/pull/35365) - fix(router): redact fallback tracebacks at the call site and cover the sync deferred stream by [@yassin-berriai](https://github.com/yassin-berriai) in [#35843](https://github.com/BerriAI/litellm/pull/35843) - fix(migrations): recover from an interrupted Prisma toolchain install by [@yassin-berriai](https://github.com/yassin-berriai) in [#35832](https://github.com/BerriAI/litellm/pull/35832) - fix(lint): bring basedpyright rule counts back under their budget limits by [@mateo-berri](https://github.com/mateo-berri) in [#35962](https://github.com/BerriAI/litellm/pull/35962) - chore(ui): don't zero out stale headroom except no-console by [@mateo-berri](https://github.com/mateo-berri) in [#35964](https://github.com/BerriAI/litellm/pull/35964) - fix(proxy): give proxy\_admin\_viewer read parity with proxy\_admin by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35851](https://github.com/BerriAI/litellm/pull/35851) - refactor(ui): address UI lint budget issues by refactoring UI by [@tin-berri](https://github.com/tin-berri) in [#35960](https://github.com/BerriAI/litellm/pull/35960) - fix(ci): make the env-key doc gate see get\_secret\_bool reads by [@yassin-berriai](https://github.com/yassin-berriai) in [#35833](https://github.com/BerriAI/litellm/pull/35833) - fix(caching): re-land evicted LLM client closing ([#35492](https://github.com/BerriAI/litellm/issues/35492)) atop self-healing handlers by [@mateo-berri](https://github.com/mateo-berri) in [#35870](https://github.com/BerriAI/litellm/pull/35870) - fix(proxy): keep the connected DB client when a startup health check fails by [@yassin-berriai](https://github.com/yassin-berriai) in [#35837](https://github.com/BerriAI/litellm/pull/35837) - chore(lint): remove litellm/types from the ruff lint exclusion by [@mateo-berri](https://github.com/mateo-berri) in [#35926](https://github.com/BerriAI/litellm/pull/35926) - feat(sgr): make the gateway middleware the source of truth for successful requests by [@yassin-berriai](https://github.com/yassin-berriai) in [#35717](https://github.com/BerriAI/litellm/pull/35717) - feat(auto-router): let operators replace the LLM classifier's system prompt by [@akapur99](https://github.com/akapur99) in [#35855](https://github.com/BerriAI/litellm/pull/35855) - fix(docker): bake the pip image's prisma engines at a world-readable path by [@yassin-berriai](https://github.com/yassin-berriai) in [#35976](https://github.com/BerriAI/litellm/pull/35976) - fix(auth): return 403 from the OAuth2 enterprise gate by [@yassin-berriai](https://github.com/yassin-berriai) in [#35838](https://github.com/BerriAI/litellm/pull/35838) - fix(router): keep custom model\_info across a price data reload by [@yassin-berriai](https://github.com/yassin-berriai) in [#35491](https://github.com/BerriAI/litellm/pull/35491) - fix(proxy): resolve pass-through credentials live from router deployments by [@mateo-berri](https://github.com/mateo-berri) in [#35916](https://github.com/BerriAI/litellm/pull/35916) - fix(ci): fetch only head and merge-base in lint jobs instead of every branch by [@mateo-berri](https://github.com/mateo-berri) in [#35982](https://github.com/BerriAI/litellm/pull/35982) - fix(autorouter): match CJK keyword\_tier\_rules that regex word boundaries miss by [@akapur99](https://github.com/akapur99) in [#35984](https://github.com/BerriAI/litellm/pull/35984) - feat(spend): rebuild the auto-router benchmarks backend as a per-session rollup by [@tin-berri](https://github.com/tin-berri) in [#35910](https://github.com/BerriAI/litellm/pull/35910) - refactor(ui): replace hand-rolled query-param routing with nuqs by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35871](https://github.com/BerriAI/litellm/pull/35871) - fix(docker): bake the componentized prisma engines at /opt/prisma so any uid can start by [@yassin-berriai](https://github.com/yassin-berriai) in [#35989](https://github.com/BerriAI/litellm/pull/35989) - fix(migrations): keep the toolchain heal from raising on an unreadable nodeenv cache by [@yassin-berriai](https://github.com/yassin-berriai) in [#35986](https://github.com/BerriAI/litellm/pull/35986) - fix(bedrock): sign Bedrock managed-file S3 requests with S3SigV4Auth by [@yassin-berriai](https://github.com/yassin-berriai) in [#35983](https://github.com/BerriAI/litellm/pull/35983) - chore(typing): replace Any seams with real types across responses, proxy, and provider adapters by [@mateo-berri](https://github.com/mateo-berri) in [#35809](https://github.com/BerriAI/litellm/pull/35809) - fix(ai21): resolve the documented AI21\_API\_KEY instead of a misspelled name by [@yassin-berriai](https://github.com/yassin-berriai) in [#35985](https://github.com/BerriAI/litellm/pull/35985) - fix(docker): fail the image build when the generated prisma engine paths drift off /opt/prisma by [@yassin-berriai](https://github.com/yassin-berriai) in [#35979](https://github.com/BerriAI/litellm/pull/35979) - fix(jina\_ai): resolve the documented JINA\_API\_KEY as a fallback by [@yassin-berriai](https://github.com/yassin-berriai) in [#35992](https://github.com/BerriAI/litellm/pull/35992) - fix(proxy): only treat a recoverable database outage as grounds to serve without one by [@yassin-berriai](https://github.com/yassin-berriai) in [#35864](https://github.com/BerriAI/litellm/pull/35864) - fix(ci): make every remaining CI checkout shallow by [@mateo-berri](https://github.com/mateo-berri) in [#35997](https://github.com/BerriAI/litellm/pull/35997) - fix(auto-router): stop the embedding model's context window from failing long requests by [@akapur99](https://github.com/akapur99) in [#35956](https://github.com/BerriAI/litellm/pull/35956) - fix(ci): make the env-key doc gate see bare get\_secret and get\_secret\_str reads by [@yassin-berriai](https://github.com/yassin-berriai) in [#35996](https://github.com/BerriAI/litellm/pull/35996) - fix(logging): extend secret redaction to records litellm does not emit directly by [@yassin-berriai](https://github.com/yassin-berriai) in [#35977](https://github.com/BerriAI/litellm/pull/35977) - test(utils): pin the register\_model replay test to the recorded half by [@yassin-berriai](https://github.com/yassin-berriai) in [#35994](https://github.com/BerriAI/litellm/pull/35994) - fix(ci): run every helm test suite, not just the first one per file by [@yassin-berriai](https://github.com/yassin-berriai) in [#35993](https://github.com/BerriAI/litellm/pull/35993) - ci: fail the build when a test file or Dockerfile is invoked by no job by [@yassin-berriai](https://github.com/yassin-berriai) in [#35991](https://github.com/BerriAI/litellm/pull/35991) - fix(langfuse): stop a collected httpx handler from closing a shared client by [@yassin-berriai](https://github.com/yassin-berriai) in [#35981](https://github.com/BerriAI/litellm/pull/35981) - fix(bedrock): grant bedrock:CountTokens in OIDC session policy by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#33145](https://github.com/BerriAI/litellm/pull/33145) - feat(pre-commit): save full lint output to a per-worktree log file by [@mateo-berri](https://github.com/mateo-berri) in [#36004](https://github.com/BerriAI/litellm/pull/36004) - feat(ui): match auto-router preset models against deployments' underlying model IDs by [@tin-berri](https://github.com/tin-berri) in [#35972](https://github.com/BerriAI/litellm/pull/35972) - fix(core\_helpers): map generic 'error' finish\_reason to 'stop' by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#33972](https://github.com/BerriAI/litellm/pull/33972) - fix(proxy)!: apply request-parameter checks consistently across body, path and form inputs by [@yuneng-berri](https://github.com/yuneng-berri) in [#36011](https://github.com/BerriAI/litellm/pull/36011) - fix: rebuild models\_by\_provider in add\_known\_models so cost map reloads reach wildcard expansion by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36010](https://github.com/BerriAI/litellm/pull/36010) - feat(complexity\_router): report LLM classifier cost per request via routing\_decision and x-litellm-classifier-cost header by [@tin-berri](https://github.com/tin-berri) in [#36015](https://github.com/BerriAI/litellm/pull/36015) - fix(model-prices): correct replicate model key typo by [@AkashNaickar](https://github.com/AkashNaickar) in [#34800](https://github.com/BerriAI/litellm/pull/34800) - fix(proxy): register managed batch output files on terminal retrieve by [@Souravrajvi0](https://github.com/Souravrajvi0) in [#34092](https://github.com/BerriAI/litellm/pull/34092) - perf(pre-commit): fetch basedpyright base counts from CI artifacts by [@mateo-berri](https://github.com/mateo-berri) in [#35970](https://github.com/BerriAI/litellm/pull/35970) - fix(ui): sync projects list page index to ?page= so back and reload keep the page by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36003](https://github.com/BerriAI/litellm/pull/36003) - fix(ui): link project page keys to their virtual key detail by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36002](https://github.com/BerriAI/litellm/pull/36002) - refactor(ui): drop unreferenced locals from dashboard route components by [@yuneng-berri](https://github.com/yuneng-berri) in [#35819](https://github.com/BerriAI/litellm/pull/35819) - fix(ui): opening a project now pushes ?project= so back and deep links work by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36001](https://github.com/BerriAI/litellm/pull/36001) - refactor(ui): drop unreferenced locals from shared dashboard components by [@yuneng-berri](https://github.com/yuneng-berri) in [#35821](https://github.com/BerriAI/litellm/pull/35821) - refactor(ui): drop unreferenced locals from tests and narrow destructures by [@yuneng-berri](https://github.com/yuneng-berri) in [#36025](https://github.com/BerriAI/litellm/pull/36025) - fix(guardrails): allow litellm\_content\_filter to run on post\_mcp\_call by [@mateo-berri](https://github.com/mateo-berri) in [#35980](https://github.com/BerriAI/litellm/pull/35980) - fix(guardrails): scan /v1/messages tool traffic by [@mateo-berri](https://github.com/mateo-berri) in [#35999](https://github.com/BerriAI/litellm/pull/35999) - refactor(ui): drop dead locals and unused React state across the dashboard by [@yuneng-berri](https://github.com/yuneng-berri) in [#36026](https://github.com/BerriAI/litellm/pull/36026) - feat(ui): add the auto-router usage tab to cost optimization by [@tin-berri](https://github.com/tin-berri) in [#35995](https://github.com/BerriAI/litellm/pull/35995) - fix(managed\_files): derive unified output file ids deterministically so concurrent registrations converge by [@mateo-berri](https://github.com/mateo-berri) in [#36019](https://github.com/BerriAI/litellm/pull/36019) - fix(proxy): send keepalive pings on anthropic messages SSE streams during upstream silence by [@mateo-berri](https://github.com/mateo-berri) in [#36024](https://github.com/BerriAI/litellm/pull/36024) - fix(managed\_files): return unified ids from unscoped file listing by [@mateo-berri](https://github.com/mateo-berri) in [#36031](https://github.com/BerriAI/litellm/pull/36031) - fix(arize\_phoenix): lowercase OTLP/gRPC auth metadata key by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#34883](https://github.com/BerriAI/litellm/pull/34883) - fix(auto-router): accept every reminder marker pair a harness emits by [@tin-berri](https://github.com/tin-berri) in [#36029](https://github.com/BerriAI/litellm/pull/36029) - fix(pricing): sync flex/priority tier keys to dated OpenAI snapshot variants by [@mateo-berri](https://github.com/mateo-berri) in [#35923](https://github.com/BerriAI/litellm/pull/35923) - fix(cost): bill reasoning tokens at the service tier output rate by [@mateo-berri](https://github.com/mateo-berri) in [#35925](https://github.com/BerriAI/litellm/pull/35925) - fix(proxy): include today's UTC bucket when a daily activity range ends at the caller's current day by [@tin-berri](https://github.com/tin-berri) in [#36051](https://github.com/BerriAI/litellm/pull/36051) - fix: expired-miss share over all measured turns + cost-optimization tab labels by [@tin-berri](https://github.com/tin-berri) in [#36037](https://github.com/BerriAI/litellm/pull/36037) - fix(router): include Bedrock batch/S3 fields and model in deployment credentials by [@mpcusack-altos](https://github.com/mpcusack-altos) in [#24548](https://github.com/BerriAI/litellm/pull/24548) - fix(batch): track cost for managed batches with no attributable key/u… by [@elinacse](https://github.com/elinacse) in [#35468](https://github.com/BerriAI/litellm/pull/35468) - feat(guardrails): add scan\_only\_tool\_results to scope unified guardrails to tool results by [@mateo-berri](https://github.com/mateo-berri) in [#36014](https://github.com/BerriAI/litellm/pull/36014) - fix(cost): stop token-pricing the placeholder input on file content calls by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35140](https://github.com/BerriAI/litellm/pull/35140) - fix(proxy): fetch background responses through the router in CheckResponsesCost by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35137](https://github.com/BerriAI/litellm/pull/35137) - fix(proxy): yaml store\_prompts\_in\_spend\_logs should take precedence over DB cached value by [@Praveena-617](https://github.com/Praveena-617) in [#35769](https://github.com/BerriAI/litellm/pull/35769) - fix(lint): measure the basedpyright budget gate in a gate-owned venv by [@mateo-berri](https://github.com/mateo-berri) in [#36050](https://github.com/BerriAI/litellm/pull/36050) - docs: cap all GitHub comments at 15-25 words, curb semicolon splices by [@mateo-berri](https://github.com/mateo-berri) in [#36059](https://github.com/BerriAI/litellm/pull/36059) - chore(lint): name MappingProxyType in the mutable-collection fix messages by [@mateo-berri](https://github.com/mateo-berri) in [#36072](https://github.com/BerriAI/litellm/pull/36072) - test: roll back runtime model registrations between tests by [@mateo-berri](https://github.com/mateo-berri) in [#36039](https://github.com/BerriAI/litellm/pull/36039) - refactor(types): cut 653 implicit and explicit Any diagnostics across 11 modules by [@mateo-berri](https://github.com/mateo-berri) in [#36054](https://github.com/BerriAI/litellm/pull/36054) - fix(proxy): stop resolving the UI session sentinel team on /search\_tools/list by [@yuneng-berri](https://github.com/yuneng-berri) in [#36061](https://github.com/BerriAI/litellm/pull/36061) - fix(batches): persist managed file ids for cancelled/failed/expired batches by [@mateo-berri](https://github.com/mateo-berri) in [#36048](https://github.com/BerriAI/litellm/pull/36048) - fix(batches): register managed output files on batch cancel by [@mateo-berri](https://github.com/mateo-berri) in [#36034](https://github.com/BerriAI/litellm/pull/36034) - fix(proxy): allow non-admins to reach /user/daily/activity/aggregated by [@yuneng-berri](https://github.com/yuneng-berri) in [#36062](https://github.com/BerriAI/litellm/pull/36062) - fix(anthropic): coerce explicit additionalProperties to false in output\_format schema by [@dkindlund](https://github.com/dkindlund) in [#35811](https://github.com/BerriAI/litellm/pull/35811) - fix(batches): prevent managed file fallbacks by [@rimysore](https://github.com/rimysore) in [#35371](https://github.com/BerriAI/litellm/pull/35371) - chore: ignore the mechanical lint and typing sweeps in git blame by [@mateo-berri](https://github.com/mateo-berri) in [#36076](https://github.com/BerriAI/litellm/pull/36076) - fix(proxy): warn at startup when max\_budget is set but no database is connected by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36041](https://github.com/BerriAI/litellm/pull/36041) - fix(proxy): promote caller metadata trace fields into litellm\_metadata by [@yucheng-berri](https://github.com/yucheng-berri) in [#35866](https://github.com/BerriAI/litellm/pull/35866) - feat(terraform): sync provider 0.3.0 from the mirror and cut 0.4.0 by [@yuneng-berri](https://github.com/yuneng-berri) in [#36098](https://github.com/BerriAI/litellm/pull/36098) - fix(guardrails): honor configured timeout in Zscaler AI Guard by [@yucheng-berri](https://github.com/yucheng-berri) in [#36110](https://github.com/BerriAI/litellm/pull/36110) - fix(logging): fall back to litellm\_metadata when metadata is empty by [@yucheng-berri](https://github.com/yucheng-berri) in [#36105](https://github.com/BerriAI/litellm/pull/36105) - fix(proxy): re-assert the authenticated identity on passthrough requests by [@yucheng-berri](https://github.com/yucheng-berri) in [#36121](https://github.com/BerriAI/litellm/pull/36121) - chore: bump litellm-enterprise 0.1.53 -> 0.1.54, litellm-proxy-extras 0.4.83 -> 0.4.84 by [@yuneng-berri](https://github.com/yuneng-berri) in [#36139](https://github.com/BerriAI/litellm/pull/36139) - fix(ui): match auto-router preset models against wildcard-expanded model groups by [@tin-berri](https://github.com/tin-berri) in [#36111](https://github.com/BerriAI/litellm/pull/36111) - test(router): assert the auto-router max\_input\_chars kwarg by [@yuneng-berri](https://github.com/yuneng-berri) in [#36109](https://github.com/BerriAI/litellm/pull/36109) - fix(ui): allow clearing a key's budget reset from the Edit Key form by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36140](https://github.com/BerriAI/litellm/pull/36140) - fix(managed\_files): skip unparseable rows when listing managed files by [@mateo-berri](https://github.com/mateo-berri) in [#36021](https://github.com/BerriAI/litellm/pull/36021) - fix(a2a): stop writing per-caller headers onto the shared cached httpx client by [@yassin-berriai](https://github.com/yassin-berriai) in [#35978](https://github.com/BerriAI/litellm/pull/35978) - build(deps): bump h2 to 4.4.1 and js-yaml to 4.3.1 by [@yuneng-berri](https://github.com/yuneng-berri) in [#36147](https://github.com/BerriAI/litellm/pull/36147) - chore: promote staging to main by [@mateo-berri](https://github.com/mateo-berri) in [#36057](https://github.com/BerriAI/litellm/pull/36057) - fix(azure\_sentinel): respect AZURE\_AUTHORITY\_HOST and derive the Azure Monitor audience per cloud by [@yucheng-berri](https://github.com/yucheng-berri) in [#36137](https://github.com/BerriAI/litellm/pull/36137) - fix(bedrock): pass SSE-KMS key through to the batch input-file S3 upload by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35148](https://github.com/BerriAI/litellm/pull/35148) - fix(anthropic adapter): stop indexing choices\[0] on choiceless streaming chunks by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35314](https://github.com/BerriAI/litellm/pull/35314) - fix(bedrock): normalize /v1/completions and /v1/responses batch records by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35675](https://github.com/BerriAI/litellm/pull/35675) - fix(proxy): return the real status code when a credential update is rejected by [@yucheng-berri](https://github.com/yucheng-berri) in [#36166](https://github.com/BerriAI/litellm/pull/36166) - fix(proxy): improve Headroom /v1/compress HTTP 404 diagnostics by [@aayush598](https://github.com/aayush598) in [#35952](https://github.com/BerriAI/litellm/pull/35952) - fix(proxy): invalidate cached project object on project update and delete by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36028](https://github.com/BerriAI/litellm/pull/36028) - feat(proxy): add apply\_user\_budget\_to\_team\_keys opt-in by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36102](https://github.com/BerriAI/litellm/pull/36102) - fix(proxy): stop alerting on health probes that lose the planned engine-restart race by [@yassin-berriai](https://github.com/yassin-berriai) in [#36141](https://github.com/BerriAI/litellm/pull/36141) - test(docker): gate the componentized gateway and backend images on an arbitrary-uid offline boot by [@yassin-berriai](https://github.com/yassin-berriai) in [#36136](https://github.com/BerriAI/litellm/pull/36136) - fix(http): stop pooled clients persisting cookies on the aiohttp jar too by [@yassin-berriai](https://github.com/yassin-berriai) in [#36149](https://github.com/BerriAI/litellm/pull/36149) - fix(router): bound fallback-walk work and error-log volume by [@yassin-berriai](https://github.com/yassin-berriai) in [#36148](https://github.com/BerriAI/litellm/pull/36148) - ci: wire credential\_endpoints tests into the proxy endpoints job by [@cursor](https://github.com/cursor)\[bot] in [#36187](https://github.com/BerriAI/litellm/pull/36187) - docs(keys): document /key/info fields and clarify budget\_reset\_at is the next reset by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36127](https://github.com/BerriAI/litellm/pull/36127) - fix(azure\_sentinel): add AZURE\_SENTINEL\_AUTHORITY\_HOST as a Sentinel scoped override by [@yucheng-berri](https://github.com/yucheng-berri) in [#36165](https://github.com/BerriAI/litellm/pull/36165) - docs(pr-template): add a User Flow section with authoring instructions by [@mateo-berri](https://github.com/mateo-berri) in [#36162](https://github.com/BerriAI/litellm/pull/36162) - fix(proxy): derive config agent ids from agent\_name so grants survive secret rotation by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36020](https://github.com/BerriAI/litellm/pull/36020) - chore(ui): regenerate schema.d.ts for the /key/info docstring update by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36210](https://github.com/BerriAI/litellm/pull/36210) - build(deps): bump gitpython to 3.1.58 to clear osv-scan on staging by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36212](https://github.com/BerriAI/litellm/pull/36212) - fix(proxy): deny agent access when key and team grants resolve to nothing by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36221](https://github.com/BerriAI/litellm/pull/36221) - build(deps): defer the second pypdf advisory until the 6.15.0 bump by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36218](https://github.com/BerriAI/litellm/pull/36218) - fix(a2a): align agent list annotation and test with the tuple return type by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36217](https://github.com/BerriAI/litellm/pull/36217) - ci: always run the UI API types sync check so it can be required by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36213](https://github.com/BerriAI/litellm/pull/36213) - build(deps): bump nanoid to 3.3.17 in the dashboard lockfile by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36227](https://github.com/BerriAI/litellm/pull/36227) - feat(ui): show user email or alias in usage data export by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36232](https://github.com/BerriAI/litellm/pull/36232) - feat(auto-router): track turns per complexity tier (LIT-5302) by [@tin-berri](https://github.com/tin-berri) in [#36209](https://github.com/BerriAI/litellm/pull/36209) - fix(websearch): restore snippet text in native web\_search\_tool\_result blocks (LIT-5315) by [@tin-berri](https://github.com/tin-berri) in [#36228](https://github.com/BerriAI/litellm/pull/36228) - fix(proxy): resolve entity access groups in the model listing endpoints by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36230](https://github.com/BerriAI/litellm/pull/36230) - fix(ui): let access groups be a team's only model source, with hover provenance by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#36234](https://github.com/BerriAI/litellm/pull/36234) - fix(managed\_files): return unified output file ids from GET /batches by [@mateo-berri](https://github.com/mateo-berri) in [#36049](https://github.com/BerriAI/litellm/pull/36049) - test(proxy): compare empty agent list to the tuple get\_agent\_list returns by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36225](https://github.com/BerriAI/litellm/pull/36225) - fix(otel): name the RPC system and upstream on MCP tool-call spans by [@yucheng-berri](https://github.com/yucheng-berri) in [#35857](https://github.com/BerriAI/litellm/pull/35857) - fix(guardrails): chunk oversized Bedrock ApplyGuardrail requests instead of failing by [@yucheng-berri](https://github.com/yucheng-berri) in [#36119](https://github.com/BerriAI/litellm/pull/36119) - test(e2e): settle control-plane writes across every replica, not just one by [@yuneng-berri](https://github.com/yuneng-berri) in [#36247](https://github.com/BerriAI/litellm/pull/36247) - fix(responses): forward allowed\_openai\_params through the chat completions bridge by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35885](https://github.com/BerriAI/litellm/pull/35885) - test(proxy): assert the copy \_add\_team\_member\_budget\_table returns by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36244](https://github.com/BerriAI/litellm/pull/36244) - chore(ui): regenerate dashboard api types for tier\_turns by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36243](https://github.com/BerriAI/litellm/pull/36243) - refactor(types): declare mirrored pricing fields on ModelInfo by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36215](https://github.com/BerriAI/litellm/pull/36215) - fix(lint): make strict-gate noqas survive base ruff and flag stale ones by [@mateo-berri](https://github.com/mateo-berri) in [#36257](https://github.com/BerriAI/litellm/pull/36257) - fix(vertex\_ai): surface real error/status on vertex batch create instead of IndexError 500 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35141](https://github.com/BerriAI/litellm/pull/35141) - ci: give the remaining pull\_request workflows a concurrency group by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36252](https://github.com/BerriAI/litellm/pull/36252) - refactor(lint): graduate zero-violation strict rules and guard the budget ratchet by [@mateo-berri](https://github.com/mateo-berri) in [#36161](https://github.com/BerriAI/litellm/pull/36161) - fix(proxy): enforce require\_managed\_files on every route that accepts a raw provider id by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35551](https://github.com/BerriAI/litellm/pull/35551) - chore(typing): clear 1.4k basedpyright Any errors across 21 hotspot files by [@mateo-berri](https://github.com/mateo-berri) in [#36282](https://github.com/BerriAI/litellm/pull/36282) - test: roll back live router replay membership between tests by [@mateo-berri](https://github.com/mateo-berri) in [#36278](https://github.com/BerriAI/litellm/pull/36278) - chore(ci): sync main into internal staging by [@yuneng-berri](https://github.com/yuneng-berri) in [#36288](https://github.com/BerriAI/litellm/pull/36288) - build(lint): rename make pre-commit to make check with a working-tree fallback by [@mateo-berri](https://github.com/mateo-berri) in [#36277](https://github.com/BerriAI/litellm/pull/36277) - fix(ui): show team BYOK models in team fallback settings by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36241](https://github.com/BerriAI/litellm/pull/36241) - fix(otel): mark v2 server spans as failed for pre-call errors by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#34546](https://github.com/BerriAI/litellm/pull/34546) - fix(websearch\_interception): bill intercepted searches to the calling key by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35708](https://github.com/BerriAI/litellm/pull/35708) - chore: remove pre-commit rule by [@mateo-berri](https://github.com/mateo-berri) in [#36295](https://github.com/BerriAI/litellm/pull/36295) - docs: clarify guideline priority ordering in CLAUDE.md by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#36296](https://github.com/BerriAI/litellm/pull/36296) - feat(router): independent, default-on deployment affinity for the auto-router by [@tin-berri](https://github.com/tin-berri) in [#36146](https://github.com/BerriAI/litellm/pull/36146) - test: repair stale CircleCI contracts by [@yuneng-berri](https://github.com/yuneng-berri) in [#36293](https://github.com/BerriAI/litellm/pull/36293) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#36286](https://github.com/BerriAI/litellm/pull/36286) - chore: rebuild Admin UI bundle for the 2026-08-08 release by [@yuneng-berri](https://github.com/yuneng-berri) in [#36297](https://github.com/BerriAI/litellm/pull/36297) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#36304](https://github.com/BerriAI/litellm/pull/36304) ##### New Contributors - [@rimysore](https://github.com/rimysore) made their first contribution in [#35367](https://github.com/BerriAI/litellm/pull/35367) - [@AkashNaickar](https://github.com/AkashNaickar) made their first contribution in [#34800](https://github.com/BerriAI/litellm/pull/34800) - [@Souravrajvi0](https://github.com/Souravrajvi0) made their first contribution in [#34092](https://github.com/BerriAI/litellm/pull/34092) - [@elinacse](https://github.com/elinacse) made their first contribution in [#35468](https://github.com/BerriAI/litellm/pull/35468) - [@aayush598](https://github.com/aayush598) made their first contribution in [#35952](https://github.com/BerriAI/litellm/pull/35952) - [@cursor](https://github.com/cursor)\[bot] made their first contribution in [#36187](https://github.com/BerriAI/litellm/pull/36187) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.96.0...v1.97.0> ### [`v1.97.0`](https://github.com/BerriAI/litellm/releases/tag/v1.97.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.96.2...v1.97.0) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.97.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.97.0/cosign.pub \ ghcr.io/berriai/litellm:v1.97.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - feat(proxy): resolve Cursor thinking/fast model-name suffixes on /cursor/chat/completions by [@mateo-berri](https://github.com/mateo-berri) in [#35554](https://github.com/BerriAI/litellm/pull/35554) - fix(team-callbacks): actually stop logging when disable\_logging is called by [@yucheng-berri](https://github.com/yucheng-berri) in [#35520](https://github.com/BerriAI/litellm/pull/35520) - refactor(lint): drop redundant !s f-string conversion flags and fix displaced import-group comments by [@mateo-berri](https://github.com/mateo-berri) in [#35546](https://github.com/BerriAI/litellm/pull/35546) - fix(proxy): backfill null user\_email on existing users during JWT auth by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#34588](https://github.com/BerriAI/litellm/pull/34588) - feat(playground): add non-streaming response toggle by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35560](https://github.com/BerriAI/litellm/pull/35560) - feat(teams): apply default organization to new teams from default team settings by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35540](https://github.com/BerriAI/litellm/pull/35540) - fix(ui): block Playground page for viewer roles on direct URL access by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35676](https://github.com/BerriAI/litellm/pull/35676) - fix(caching): close evicted LLM clients so their connections are reclaimed by [@yassin-berriai](https://github.com/yassin-berriai) in [#35492](https://github.com/BerriAI/litellm/pull/35492) - chore(deps): update brace-expansion, postcss, and gitpython to current patch releases by [@yuneng-berri](https://github.com/yuneng-berri) in [#35692](https://github.com/BerriAI/litellm/pull/35692) - refactor(ui): rename the create MCP server component to PascalCase by [@yuneng-berri](https://github.com/yuneng-berri) in [#35686](https://github.com/BerriAI/litellm/pull/35686) - fix(openai): drop undefined Union from owns\_wrapped\_http\_client annotation by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35706](https://github.com/BerriAI/litellm/pull/35706) - fix(openai): drop the undefined Union from owns\_wrapped\_http\_client by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35704](https://github.com/BerriAI/litellm/pull/35704) - chore(ui): note Google's Agent Platform rename in vector store setup by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28076](https://github.com/BerriAI/litellm/pull/28076) - fix(proxy): apply key/team router\_settings.model\_group\_alias by [@yassin-berriai](https://github.com/yassin-berriai) in [#35486](https://github.com/BerriAI/litellm/pull/35486) - feat(complexity\_router): default session affinity off and expose it in the UI by [@tin-berri](https://github.com/tin-berri) in [#35714](https://github.com/BerriAI/litellm/pull/35714) - fix(datadog): read team callback dd\_\* params from kwargs instead of blocked dynamic params ([#35115](https://github.com/BerriAI/litellm/issues/35115) port) by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35687](https://github.com/BerriAI/litellm/pull/35687) - refactor(ui): extract the MCP create form's logic and field groups by [@yuneng-berri](https://github.com/yuneng-berri) in [#35694](https://github.com/BerriAI/litellm/pull/35694) - test(ui): tier the MCP create tests into unit and integration by [@yuneng-berri](https://github.com/yuneng-berri) in [#35697](https://github.com/BerriAI/litellm/pull/35697) - fix(proxy): redact credential headers from request logging copies by [@yucheng-berri](https://github.com/yucheng-berri) in [#35678](https://github.com/BerriAI/litellm/pull/35678) - feat(guardrails/rubrik): prompt moderation, response-text blocking, streaming buffer, failure logging by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35722](https://github.com/BerriAI/litellm/pull/35722) - fix(ui): render Responses API request and response in the logs drawer by [@yuneng-berri](https://github.com/yuneng-berri) in [#35718](https://github.com/BerriAI/litellm/pull/35718) - fix(ui): hide guardrail review buttons from non-admin users by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#27535](https://github.com/BerriAI/litellm/pull/27535) - feat(team): custom metadata validation hook for team create and update by [@yuneng-berri](https://github.com/yuneng-berri) in [#33353](https://github.com/BerriAI/litellm/pull/33353) - ci(circleci): install a pinned Rust toolchain on the Linux jobs by [@yuneng-berri](https://github.com/yuneng-berri) in [#35519](https://github.com/BerriAI/litellm/pull/35519) - fix(bedrock): stop forwarding no-op toolSpec.strict to Converse by [@tin-berri](https://github.com/tin-berri) in [#35688](https://github.com/BerriAI/litellm/pull/35688) - fix(ui): reject an auto-router keyword rule left empty instead of dropping it by [@tin-berri](https://github.com/tin-berri) in [#35705](https://github.com/BerriAI/litellm/pull/35705) - fix(guardrails/rubrik): attribute blocked requests to the caller that made them by [@yucheng-berri](https://github.com/yucheng-berri) in [#35734](https://github.com/BerriAI/litellm/pull/35734) - fix(responses): forward client headers to the provider on /v1/responses by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#34531](https://github.com/BerriAI/litellm/pull/34531) - feat(spend): add net auto-router savings to the cost-optimization dashboard by [@tin-berri](https://github.com/tin-berri) in [#35521](https://github.com/BerriAI/litellm/pull/35521) - chore(typing): clear basedpyright Any errors in budget reset, access groups, and cache settings by [@mateo-berri](https://github.com/mateo-berri) in [#35719](https://github.com/BerriAI/litellm/pull/35719) - fix(spend): read what a request cost from the record instead of pricing it again by [@tin-berri](https://github.com/tin-berri) in [#35736](https://github.com/BerriAI/litellm/pull/35736) - perf: install hiredis so redis-py parses replies with its C parser by [@Classic298](https://github.com/Classic298) in [#35709](https://github.com/BerriAI/litellm/pull/35709) - feat(ui): show auto-router savings on the cost-optimization dashboard by [@tin-berri](https://github.com/tin-berri) in [#35522](https://github.com/BerriAI/litellm/pull/35522) - perf: build log messages lazily so filtered-out log records cost nothing by [@Classic298](https://github.com/Classic298) in [#35703](https://github.com/BerriAI/litellm/pull/35703) - fix(proxy): retry model cost map fetch with Retry-After-aware backoff and keep current map on reload failure by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#35739](https://github.com/BerriAI/litellm/pull/35739) - feat(otel): stamp service tier attributes on inference spans by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#35679](https://github.com/BerriAI/litellm/pull/35679) - fix(proxy): log the model cost map reload failure lazily by [@tin-berri](https://github.com/tin-berri) in [#35750](https://github.com/BerriAI/litellm/pull/35750) - fix(groq): translate web\_search\_options to the browser\_search tool by [@hMED22](https://github.com/hMED22) in [#34971](https://github.com/BerriAI/litellm/pull/34971) - feat(ui): add admin-configurable user banner by [@yuneng-berri](https://github.com/yuneng-berri) in [#35729](https://github.com/BerriAI/litellm/pull/35729) - fix(e2e): make spend-counter redis connection env-driven for non-cluster deployments by [@yuneng-berri](https://github.com/yuneng-berri) in [#35732](https://github.com/BerriAI/litellm/pull/35732) - fix(proxy): make /cursor/chat/completions work with Cursor agent mode by [@tin-berri](https://github.com/tin-berri) in [#34029](https://github.com/BerriAI/litellm/pull/34029) - fix(proxy): propagate user\_email and bind api\_key on JWT auth attribution paths by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#34331](https://github.com/BerriAI/litellm/pull/34331) - chore(build): move the Admin UI toolchain to Node 24 by [@yuneng-berri](https://g…

tin-berri requested a review from a team August 1, 2026 23:06

tin-berri mentioned this pull request Aug 1, 2026

feat(ui): show auto-router savings on the cost-optimization dashboard #35522

Merged

5 tasks

greptile-apps Bot reviewed Aug 1, 2026

View reviewed changes

Comment thread litellm/router_strategy/savings_baseline.py Outdated

tin-berri added 2 commits August 1, 2026 16:51

tin-berri force-pushed the litellm_autorouter_savings_backend branch from 6c72dda to c6e0f1b Compare August 1, 2026 23:53

cursor Bot reviewed Aug 2, 2026

View reviewed changes

Comment thread litellm/router_strategy/complexity_router/complexity_router.py Outdated

Comment thread litellm/proxy/spend_tracking/savings.py

cursor Bot reviewed Aug 2, 2026

View reviewed changes

Comment thread litellm/proxy/spend_tracking/savings.py Outdated

cursor Bot reviewed Aug 2, 2026

View reviewed changes

greptile-apps Bot reviewed Aug 2, 2026

View reviewed changes

Comment thread litellm/router_strategy/savings_baseline.py Outdated

veria-ai Bot reviewed Aug 2, 2026

View reviewed changes

test(spend): follow _most_expensive onto the router that prices its c…

6e17303

…andidates Ranking moved through `Router.get_deployment_model_info`, since what a deployment costs is the router's answer to give; these four cases were still calling the old free-function signature.

greptile-apps Bot reviewed Aug 3, 2026

View reviewed changes

Comment thread litellm/router_strategy/savings_baseline.py Outdated

cursor Bot reviewed Aug 3, 2026

View reviewed changes

Comment thread litellm/proxy/spend_tracking/savings.py

cursor Bot reviewed Aug 3, 2026

View reviewed changes

github-actions Bot mentioned this pull request Aug 3, 2026

🦞 OpenClaw 生态日报 2026-08-04 junlinzhao327-oss/big_model_radar#251

Open

cursor Bot reviewed Aug 4, 2026

View reviewed changes

mateo-berri approved these changes Aug 4, 2026

View reviewed changes

yucheng-berri approved these changes Aug 4, 2026

View reviewed changes

tin-berri force-pushed the litellm_autorouter_savings_backend branch from 8499c17 to e5e0396 Compare August 4, 2026 02:46

tin-berri enabled auto-merge (squash) August 4, 2026 03:04

tin-berri merged commit 9e3a8df into litellm_internal_staging Aug 4, 2026
80 of 81 checks passed

tin-berri deleted the litellm_autorouter_savings_backend branch August 4, 2026 03:10

This was referenced Aug 4, 2026

fix(spend): read what a request cost from the record instead of pricing it again #35736

Merged

feat(spend): add a per-session auto-router benchmarks rollup #35839

Open

feat(spend): derive a default auto-router savings baseline from the hardest tier #35907

Merged

devin-ai-integration Bot mentioned this pull request Aug 5, 2026

docs(auto_routing): update the setup skill and page for tier labels, classifier prompt, and fallback BerriAI/litellm-docs#787

Merged

Uh oh!

Uh oh!

Conversation

tin-berri commented Aug 1, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

Relevant issues

Linear ticket

Pre-Submission checklist

Screenshots / Proof of Fix

Type

Changes

Final Attestation

Uh oh!

greptile-apps Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

Uh oh!

codecov Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tin-berri commented Aug 1, 2026

Uh oh!

Uh oh!

Uh oh!

codspeed-hq Bot commented Aug 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

tin-berri commented Aug 2, 2026

Uh oh!

tin-berri commented Aug 2, 2026

Uh oh!

Uh oh!

tin-berri commented Aug 2, 2026

Uh oh!

tin-berri commented Aug 2, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

veria-ai Bot Aug 2, 2026

Choose a reason for hiding this comment

Uh oh!

tin-berri Aug 2, 2026

Choose a reason for hiding this comment

Uh oh!

veria-ai Bot Aug 2, 2026

Choose a reason for hiding this comment

Uh oh!

veria-ai Bot commented Aug 2, 2026

PR overview

Open issues (1)

Uh oh!

tin-berri commented Aug 3, 2026

Uh oh!

tin-berri commented Aug 3, 2026

Uh oh!

Uh oh!

Uh oh!

tin-berri commented Aug 3, 2026

Uh oh!

greptile-apps Bot commented Aug 3, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

tin-berri commented Aug 3, 2026

Uh oh!

tin-berri commented Aug 3, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

tin-berri commented Aug 1, 2026 •

edited by cursor Bot

Loading

greptile-apps Bot commented Aug 1, 2026 •

edited

Loading

codecov Bot commented Aug 1, 2026 •

edited

Loading

codspeed-hq Bot commented Aug 2, 2026 •

edited

Loading